Gestural Control of Household Appliances for the Physically Impaired

Household appliances such as dishwashers, televisions and radios are an indispensable part of the modern household. Yet, people who have some form of physical impairment often find that they are unable to make use of these commonly available appliances, to the detriment of their lifestyle. This paper proposes a gesture interface for home appliances that can be used by people with physical impairments. Two simulated gesture controlled appliances are developed and evaluated by physically impaired people. The results show that this interface is able to allow physically impaired people to make use of modern appliances by gesture. Introduction and Motivation The ability to live a satisfying, independent life can be difficult for people who are physically impaired. While there are tools and technology that can assist physically impaired people in various ways, there are few practical solutions to assist with the operation of home appliances. Modern home appliances provide a great deal of convenience, comfort and information to our lives. We depend on appliances such as dishwashers, ovens, televisions and vacuum cleaners to assist with keeping our homes clean, preparing meals and for information and entertainment. While most people give little thought to the physical control of an appliance when operating it, there are some people who often find that they either cannot use or require some form of assistance to use modern appliances due to the physical impairments they have. For example, a person confined to a wheelchair will have difficulties in reaching for appliance controls, especially if they have spinal muscular problems that prevent or severely restrict upper body movement. In this paper, we explore the use of gesture recognition as a method for physically impaired people to control home appliances or any device in their environment through investigation and development of a gesture recognition system and a set of easy-to-use gestures. The goal of this work is to design and demonstrate a gesture interface that will provide people with physical impairments the ability to use common home appliances. To achieve this goal, the gesture interface will need to meet the following objectives: Copyright c © 2012, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. Invisible The interface should not be the focus of the user’s attention when controlling the appliance. The user should feel as if they are controlling the appliance itself. It ideally should be embedded within the appliance and not be an external, discrete device. Simple The set of gestures should be simple enough to be used by any person and with any appliance. Simple gestures will accommodate a wider range of impairment and ability. Intuitive Gestures should be relevant to the functions they control, and the user should directly interact with the desired appliance. Ambient The appliance and interface can be ignored until it is explicitly required. Also, the appliance will not respond or activate until needed. Gesture-Based User Interface Ambient Intelligent (AmI) systems have been identified as a possible way of home automation. The usual methods for user interaction proposed for AmI systems is a combination of speech recognition and environmental sensing through cameras, motion sensors and similar devices. Recently, the use of cell phones, PDAs, multi-touch screens and, last but not least, gesture interfaces (Sainz Salces et al. 2006; Starner et al. 2000; Raisinghani et al. 2004; Magerkurth et al. 2006) has been advocated as alternatives to interaction with an AmI system. Considering that direct-manipulation interface devices, such as light pens, have been found to be easier to use (Charness, Bosman, and Elliot 1995), it would seem that a gesture interface system would also be similarly useful for people with physical impairments, since gestures are an integral part of communication and are developed at an early age before the acquisition of language skills (Butcher and Goldin-Meadow 2000). There are a number of different methods to implement a gesture system, including the use of plush toys (Mori and Igarashi 2007), whole body movement (Bien et al. 2005), eye winks (Wei-Gang, Huang, and Hwang 2007), hand movements (Starner et al. 2000; Sepehri, Yacoob, and Davis 2006) and the use of a remote “wand” (Wilson and S. Shafer 2003; Wobbrock, Wilson, and Li 2007). The problem with all of these approaches is that they require a customised hand-held device and/or fixed cameras in the envi353 Proceedings of the Twenty-Fifth International Florida Artificial Intelligence Research Society Conference